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ABSTRACT 

This study examined the impact of mid-semester student ratings feedback on a faculty’s end-of- 
semester student ratings. The positive direction of the end-of-semester ratings in the two mid- 
semester feedback groups lent support to the premise that cognitive dissonance theory and 
various forms of mid-semester, student rating feedback can be used to improve teaching. 
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Utilizing Cognitive Dissonance Theory to Improve 
Student Ratings of College Faculty Members 

Formal student evaluations are conducted at most colleges for both full-time and part- 
time faculty members with the assumption that instructors will use this information to improve 
their teaching (Centra, 1973). Some studies have indicated that student feedback has had a 
positive effect on teaching performance and that student ratings are moderately correlated with 
student learning (Centra, 1973, 1993). Overall, student ratings have also been shown to improve 
significantly when faculty receive augmented feedback (Brinko, 1990; Cohen, 1980). Cohen 
(1980) recommended that more studies needed to be done to verify the impact of student ratings 
on teaching behavior. This study, based on an experiment by Centra (1973), addressed using 
student ratings to improve teaching effectiveness as measured by student ratings of university 
faculty members. This study examined using not only student ratings but also augmenting 
student ratings with faculty consultation to improve teaching effectiveness. 

The experiment by Centra (1973) was based on the theory of cognitive dissonance. 

When teaching was evaluated at mid-semester less favorably than the faculty member expected, 
it was presumed that faculty members valued the opinions of their students enough to change 
their instructional methods at mid-semester to be more responsive to students’ needs (Centra, 
1973). Several theories provide justification for this belief (Daw & Gage, 1967; Festinger, 1957; 
Gage, Runkle & Chaterjee, 1963; Heider, 1958; Newcomb, 1959). The theory of cognitive 
dissonance holds that when a discrepancy exists between an individual’s perceptions and 
behavior, the individual seeks to make the perceptions and the behaviors more congruent, 
thereby resolving the dissonance (Festinger, 1957). 
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With the application of this theory to this study, the ratings given by the students at mid- 
semester, the instructors’ self-ratings, the discrepancy scores, the written feedback, and the 
consultation workshop were expected to influence or explain the changes observed on the overall 
teaching effectiveness measure. This means that the higher end-of-semester student ratings 
should have reflected improvements in teaching behaviors. If the students’ mid-semester ratings 
were lower than the mid-semester self-ratings of the faculty members, the faculty members were 
expected to improve their teaching behavior in order to be more congruent with their students’ 
higher expectations. 

One of the purposes of this experimental study was to test again the theory of cognitive 
dissonance that compared discrepancies in student and faculty ratings to changes in faculty 
members’ teaching behavior as measured by end of the semester student ratings. Other purposes 
for this study were to recommend: (a) new policies and procedures for faculty evaluation systems 
and (b) additional needed research. The research question in this study was: What are the effects 
of written feedback and group consultation on faculty-specific teaching behaviors as measured 
by student ratings? (Carson, 1999). The null hypothesis for this study was: Having statistically 
controlled for student motivation, there is no statistically significant mean difference among the 
three groups of instructors with respect to end of semester student ratings. 

Method 

The Sample 

All 158 faculty members at a small, private college were invited in December of 1998 to 
participate in this study by their Vice President for Academic Affairs. Approximately 73 faculty 
members at this college were full-time and approximately 85 were part-time. Sixty-one faculty 
members volunteered to participate in this study (59 full-time faculty members and two part-time 
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faculty members). Random selection, based on a table of random numbers, was used to assign 
20 volunteers to each of the three groups (with the consultation group having one extra 
instructor). 

To test the hypothesis for this study the following three groups were used: 

1. In one faculty group, identified as the control group, faculty members were rated by the 
students at the end of the semester. This group received only written feedback after the 
end of the semester. 

2. In the first treatment group, identified as the feedback group, faculty members rated 
themselves at mid-semester and were rated by students at mid-semester. This first 
treatment group received prompt, mid-semester written feedback on discrepancies 
between their ratings and their students’ ratings. At the end of the semester, they were 
rated again by their students. 

3. In a second treatment group, identified as the consultation group, faculty members rated 
themselves at mid-semester and were rated by students at mid-semester. The faculty 
members in this group received prompt, mid-semester written feedback on their ratings 
and their students’ ratings. In addition to the prompt feedback, they received a one-hour, 
group consultation (workshop) with their feedback. At the end of the semester, they were 
rated again by their students. 

Variables and their Measurement 

The levels of the experimental/independent variable included: 1) the written feedback 
provided to the feedback group, 2) the group consultation provided in conjunction with the 
written feedback to the consultation group, and 3) the control group. The five dependent 
variables were the changes in the teaching effectiveness (behavior) as measured by changes in 
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the student ratings at mid-semester compared to those ratings at the end of the semester as 
measured by the student rating instrument. Discrepancy scores were computed for each faculty 
member by taking the average of the students’ ratings on a specific outcome variable and then 
subtracting that figure from the faculty’s own ratings on the same outcome variable. The 
researchers were aware of the problems typically found when dealing with this type of score, that 
is: low reliability, restricted range, and issues involving response set. These limitations on 
discrepancy scores were partially overcome by the application of appropriate statistical 
procedures, such as: analysis of covariance and repeated-measures of ANOVA (Gall, Borg, & 
Gall, 1996). The covariate variable used here was the item dealing with students’ motivation as 
described in the student rating instrument. 

Instrumentation 

The staff of the Individual Development and Educational Assessment (IDEA) Center at 
Kansas State University (KSU) focuses on the assessment and improvement of teaching and 
learning at colleges and universities. The instrument used in this study for instructor self and 
student ratings, the IDEA Student Rating of Instruction System (long form), was designed to 
improve student learning (Hoyt & Cashin, 1977). There are 47 items on this questionnaire 
designed to measure teaching behaviors; however, only responses to the first 20 items were used 
in this investigation plus the motivation item. These items follow a Likert-type format with a 
code that included a “hardly ever” response to an “almost always” frequency response. 

Reliability and validity studies have been conducted for the IDEA instrument in 
university teaching, (Hoyt & Cashin, 1977; Marsh, 1984, Marsh, 1994). Most studies provided 
acceptable indices of internal consistency and relevant confirmatory information about the 
dimensionality of the instrument. More specifically, the reliability of the IDEA Survey Form for 
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student ratings had been previously established through a correlation study (Hoyt & Cashin, 
1977). The estimated reliabilities ranged from .81 to .94, and averaged .87. Standard errors of 
measurement averaged about 0.3. The validity of the IDEA Survey Form had also been 
previously established in a number of ways (Hoyt & Cashin, 1977). The IDEA survey asks 
students to give the same kind of ratings as other student rating systems. Thus, the validity of 
those parallel systems had been applied as well to the IDEA system. Further validity was 
established by validating certain aspects that are unique to the IDEA system including: 

(a) students’ reports of progress, (b) the relationships between teaching methods and objectives, 
and (c) the influence of class size and student motivation. 

Analyses of Data 

Hoyt and others (1999) have grouped these 20 teaching behaviors (items 1-20 on the 
IDEA Survey Form) into the following five factors: (a) student-faculty contact (items 1, 2, 20); 

(b) involving students (items 5, 9, 14, 16, 18); (c) establishing expectations (items 3, 4, 8, 13, 

15); (d) clarity of communication (items 6, 10, 11); and (e) assessment/feedback (items 7, 12, 17, 
19). The 20 items were grouped into these five factors for analysis on the IDEA Reports. These 
same five factors were used to represent the five dependent variables (teaching behaviors) in the 
data analyses for this study. 

The study hypothesis was analyzed with a multivariate analysis of covariance (Glass & 
Hopkins, 1996; Harris, 1975; Popham & Sirontnik, 1992). The control variable was student 
motivation and was measured by item 39 on the IDEA Survey Form which reads, “I really 
wanted to take this course no matter who taught it.” In addition to the IDEA system, the 
researchers developed a discrepancy feedback sheet. These sheets were utilized to provide 
written feedback to faculty members. This feedback sheet consisted of the 20 Teaching Method 
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items from the IDEA system. The class average on each item, the instructor’s self-rating on each 
item, and the calculated discrepancy scores on each item were listed on this sheet which was 
given to the faculty members in the feedback and consultation groups at mid-semester. 

Collection of Data 

In the spring of 1999, one researcher worked with the administrative assistant to the Vice 
President for Academic Affairs to obtain the student evaluations from the three separate groups 
at mid-semester and at the end of the semester at the college. The IDEA evaluation forms were 
processed by the IDEA Center at KSU in Manhattan, Kansas. The instructors’ self-rating forms 
were processed by the researcher who calculated the 20 discrepancy scores for each of the 39 
faculty members in the two experimental groups. 

The evaluation forms were handled in the same manner as the other student ratings at this 
college. The forms were picked up by the faculty member. Then a student was selected by the 
faculty member to distribute the forms in class without the instructor present. The student 
collected the rating sheets after they were completed. The student then returned the forms in a 
sealed envelope to the vice president’s office. Once all of the forms were returned, they were 
packed and shipped by the researcher to the IDEA Center for processing. The same process was 
followed at both mid-semester and at the end of the semester. 

Results 

Table 1 contains the descriptive statistics used in the between group comparisons. The 
descriptive statistics used in the between group comparisons included the means of each group 
on each of the five student rating factors. These means were calculated individually for each 
instructor by using his or her individual items scores from the items associated with the 
appropriate factor and dividing these scores by the appropriate number of students. Thus each 
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instructor in the three groups had five individual average scores on the five factors. These factor 
scores were then averaged for each group resulting in means for each of these factors for each 
group. 

The multivariate test, the Wilks’ Lambda , was used to analyze the data for the between 
groups hypothesis. Student motivation was the control variable used in this analysis. Table 2 
outlines the results of this multivariate test. As indicated on Table 2, the results did not reflect 
the anticipated significant differences between the end-of-the-semester student ratings of the 
groups. The student motivation covariate did result in significance, but marginally. This 
significance had been expected based on the previous literature review (Centra, 1973). 

The results in Table 2 indicated there was a statistically significant difference between 
the three groups with respect to the covariate, student motivation; F = 2.364, p = .053. However, 
there was no statistically significant difference between the three groups of instructors with 
respect to end-of-semester student ratings, F = .359, p = .961, indicating that there was no 
apparent treatment effect across the three levels of the independent variable and the five outcome 
measures. 

Even though there was no significant difference between the experimental groups of 
instructors with respect to the end-of-the-semester student ratings, all of the means, except one, 
were consistent with the predicted positive direction (see Table 1). The only mean that did not 
reflect the positive predicted direction was the mean for the factor of “involving students”, which 
was larger for the feedback group than for the consultation group (3.57 vs. 3.53). All of the 
means (except for involving students) were greater for the workshop group than for the feedback 
group or the control group. All of the means for the feedback group were greater than those of 
the control group. This finding in itself provides some support for the conclusion that 
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Table 1. Dependent Variables Means Used in Between Groups Comparison For Hypothesis One 



Dependent 

Variables 


Group 


Dependent 

Variables 

Means 


Standard 

Deviation 


N 


Student-Faculty 


Consultation 


4.08 


.45 


20 


Contact 


Feedback 


4.06 


.48 


19 




Control 


3.98 


.43 


19 


Involving 


Consultation 


3.53 


.68 


20 


Students 


Feedback 


3.57 


.76 


19 




Control 


3.35 


.68 


19 


Establishing 


Consultation 


4.00 


.49 


20 


Expectations 


Feedback 


3.97 


.48 


19 




Control 


3.84 


.53 


19 


Clarity of 


Consultation 


4.21 


.33 


20 


Communication 


Feedback 


4.11 


.45 


19 




Control 


3.97 


.59 


19 


Assessment/ 


Consultation 


3.99 


.50 


20 


Feedback 


Feedback 


3.96 


.50 


19 




Control 


3.87 


.53 


19 



Table 2. Wilks’ Lambda Statistics For the Multivariate Analysis of Covariance 


Effect 


Value 


F 


Hypothesis df 


Error df 


P value 


Motivation 


.809 


2.364 


5 


50. 


.053 


Group 


.932 


.359 


10 


100. 


.961 
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written feedback combined with a one-hour group consultation seems to have a positive effect 
on teaching effectiveness. Additionally, t -tests on all 20 teaching behaviors were computed. 

The results supported the predicted positive direction with 17 of the 20 means of the consultation 
group higher than those of the control group, 13 of the 20 means of the consultation group higher 
than those of the feedback group, and 20 of the 20 means of the feedback group higher than 
those of the control group at the end of the spring semester. 

Discussion 

Regarding the hypothesis and controlling for student motivation, there was no statistically 
significant difference among the three groups of instructors with respect to end-of-semester 
student ratings, (see Table 2). The lack of significance could have resulted from a number of 
possible reasons. These include: (a) the extensive exposure to faculty evaluations, (b) the short 
one-semester time frame, (c) the length of the IDEA long form, (d) the sample size, (e) the 
group, rather than individual, nature of the consultations, and (f) inherent limitations of the 
student and faculty ratings due to response sets typically found in self-rating scales. 

A number of assumptions were made regarding this study. The following briefly looks at 
these assumptions in light of the findings. The findings did provide some positive support for 
the assumption that cognitive dissonance is a reliable theory with real application potential for 
improving teaching effectiveness (student ratings) among college faculty members. The positive 
means of the treatment groups indicated that feedback in general is valuable. The assumption 
that faculty members value the opinions of their students enough to change their instructional 
methods to be more responsive to students’ needs was supported to a degree based on the 
positive direction of the means on the dependent variables. The assumption that student 
feedback, augmented by group consultation, is a useful method for improving teaching 
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effectiveness among college faculty members has merit in that all of the means for this group 
reflected a positive direction. Although this study did not result in statistically significant group 
differences, Brinko’s (1990) study supported this assumption with statistically significant results. 
Brinko’s more positive results may have been due to the fact that individual consultation rather 
than group consultation was utilized in her research. 

Recommendations for further study include: (a) replicating this study over a two- 
semester period; (b) conducting this study using individual as opposed to group consultation; 

(c) conducting this study in a qualitative format with interviews with students and with 
faculty members; (d) conducting this type of study with an institution’s own instrument; and 
(e) conducting this type of study with only new faculty members. 

The implications of this study for policies and practices include the following: (a) student 
evaluations are an important means to improve teaching effectiveness, (b) self-evaluation by 
faculty members can result in positive insights to teaching effectiveness, (c) discrepancy scores 
between self and student evaluations at mid-semester can provide valuable information to faculty 
members, and (d) a workshop/consultation approach to student ratings feedback at mid-semester 
can be an effective, low-cost means to improve teaching effectiveness. 

Higher education today continues to face numerous challenges. In response to some of 
these challenges, teaching effectiveness can always be improved. One aspect of teaching 
effectiveness is improved teaching behaviors. Student evaluations are conducted by most 
institutions of higher education with the goal of improving student ratings of instructors and 
thereby improving teaching effectiveness. The positive change of student ratings in this study 
lends support to the premise that cognitive dissonance theory and mid-semester student feedback 
can be used to improve the student ratings of college teachers. 




13 



r 



Brinko, K. (1990). Instructional consultation with feedback in higher education. Journal of 
Higher Education. 61 (1). 65-83. 

Carson, R. D. (1999). Utilizing Cognitive Dissonance Theory to Improve Student Rating of 
College Faculty. Unpublished doctoral dissertation, Texas Tech University, Lubbock, 
Texas. 

Centra, J. A. (1973). Effectiveness of student feedback in modifying college instruction. Journal 
of Educational Psychology, 65 (3). 395-401. 

Centra, J. A. (1993). The development of the student instructional report II. Princeton, NJ: 
Educational Testing Service. 

Cohen, P. A. (1980). Effectiveness of student-rating feedback for improving college instruction: 
A meta-analysis of findings. Research in Higher Education, 13, 321-341. 

Daw, R. W., & Gage, N. L. (1967). Effect of feedback from teachers to principals. Journal of 
Educational Psychology, 58, 181-188. 

Festinger, L. (1957). A theory of cognitive dissonance. Evanston. IL: Row, Peterson. 

Gage, N. L., Runkle, P. J., & Chatterjee, B. B. (1963). Changing teacher behavior through 

feedback from pupils: An application of equilibrium theory. In W. W. Charters & N. L. 
Gage (Eds.). Readings in the social psychology of Education. Boston: Allyn & Bacon. 

Gall, M., Borg, W., & Gall, J. (1996). Educational research: An introduction (6th ed.). White 
Plains, NY: Longman. 

Glass, G., & Hopkins, K. (1996). Statistical methods in education and psychology, (3rd ed.). 
Needham Heights, MA: Simon & Schuster. 

Harris, R. J. (1975). A primer of multivariate statistics. New York: Academic Press. 




14 



14 



Heider, F. (1958). The psychology of interpersonal relationships. New York: Wiley. 

Hoyt, D. P., & Cashin, W. E. (1977). Development of the IDEA system (Tech. Rep. No. 1). 

Manhattan: Kansas State University, Center For Faculty Evaluation and Development. 

Hoyt, D. P., Chen, Y., Pallet, W., & Gross, A. (1999). Revising the IDEA system for obtaining 
student ratings of instructors and courses (Tech. Rep. No. 11). Manhattan, KS: Kansas 
State University, Center for Faculty Evaluation and Development. 

Marsh, H. W. (1984). Students’ evaluations of university teaching: Dimensionality, reliability, 
validity, potential biases, and utility. Journal of Educational Psychology, 76, 707-754. 

Marsh, H. W. (1994). Weighting for the right criteria in the IDEA System: Global and specific 
ratings of teaching effectiveness and their relationship to course objectives. Journal of 
Educational Psychology, 86, 631-648. 

Newcomb, T. M. (1959). Individual systems of orientation. In S. Koch (Ed.), Psychology: A 
study of science. Vol. 3. New York: McGraw-Hill 

Popham, W. J., & Sirotnik, K. A. (1992). Understanding statistics in education. Itasca, IL: F. E. 
Peacock Publishers. 



O 

ERIC 



15 



H-EV3 feDSt 



4' 




U.S. Department of Education 

Office of Educational Research and Improvement (OERI) 
National Library of Education (NLE) 
Educational Resources Information Center (ERIC) 




REPRODUCTION RELEASE 

(Specific Document) 



I. DOCUMENT IDENTIFICATION: 



Title: HI 7' - r :J r, 

[/' J ( 1 1 *4 LOjs. 44 1 1 

Rr Co((e<£sA fisycici-k/ 1 


^SSoM.ance' 


'fk.e.o <rj ~to 5 tu c)e*h/<cfftys 


Author(s): Oxi/USoh k 


£<< C A * v Miestlhs. folidaw* Avtu^tU. 


1 y ' y J 

Corporate Source: ■<" „ 1 I n « . / 

/eX-^5 l CCJ^U iOC^S [Tl q 

}^icA\ £s' "fl CPOy, if'o, vL\ 


Publication Date: fj, £ 

- $oo.) ’7 2ooo 



II. REPRODUCTIO^ELEASE: 



In order to disseminate as widely as possible timely and significant materials of interest to the educational community, documents announced in the 
monthly abstract journal of the ERIC system, Resources in Education (RIE), are usually made available to users in microfiche, reproduced paper copy, 
and electronic media, and sold through the ERIC Document Reproduction Service (EDRS). Credit is given to the source of each document, and, if 
reproduction release is granted, one of the following notices is affixed to the document. 



If permission is granted to reproduce and disseminate the identified document, please CHECK ONE of the following three options and sign at the bottom 
of the page. 



The sample sticker shown below will be 
affixed to all Level 1 documents 


The sample sticker shown below will be 
affixed to all Level 2A documents 


The sample sticker shown below will be 
affixed to all Level 2B documents 






PERMISSION TO REPRODUCE AND 






PERMISSION TO REPRODUCE AND 




DISSEMINATE THIS MATERIAL IN 




PERMISSION TO REPRODUCE AND 


DISSEMINATE THIS MATERIAL HAS 




MICROFICHE. AND IN ELECTRONIC MEDIA 




DISSEMINATE THIS MATERIAL IN 


BEEN GRANTED BY 




FOR ERIC COLLECTION SUBSCRIBERS ONLY. 




MICROFICHE ONLY HAS BEEN GRANTED BY 






HAS BEEN GRANTED BY 










0 \® 




A® 






c3y 




cJP 


TO THE EDUCATIONAL RESOURCES 




TO THE EDUCATIONAL RESOURCES 






TO THE EDUCATIONAL RESOURCES 


INFORMATION CENTER (ERIC) 




INFORMATION CENTER (ERIC) 




INFORMATION CENTER (ERIC) 


1 




2A 




2B 


Level 1 


Level 2A 


Level 2B 




i 




t 




Check here for Level 1 release, permitting reproduction Check here for Level 2A release, permitting reproduction Check here for Level 2B release, permitting 

and dissemination in microfiche or other ERIC archival and dissemination in microfiche and in electronic media reproduction and dissemination in microfiche only 

media (e.g.. electronic) and paper copy. for ERIC archival collection subscribers only 



Documents will be processed as indicated provided reproduction quality permits. 

If permission to reproduce is granted, but no box is checked, documents will be processed at Level 1 . 



Sign 

here,-* 

-<3" e 

ERIC 



/ hereby grant to the Educational Resources Information Center (ERIC) nonexclusive permission to reproduce and disseminate this document 
as indicated above. Reproduction fino/n the ERIC microfiche or electronic media by persons other than ERIC employees and its system 
contractors requires permission from ttwlcopyright tydld^r.* Exception /s)nade for nonprofit reproduction by libraries and other service agencies 
to in re 




Ported Name/Position^Title: tj j\ fN, a | 

/ (L eArli. rrokzsor £>r 


!e.dy (Jm 




^0^741-2(79 


E-^ail Address:/-\ // // 

tstk itw^hcSe-rbu., 


Da,e V/V<5/ 


fx 7fW3 r-> — 



(over) 




III. DOCUMENT AVAILABILITY INFORMATION (FROM NON-ERIC SOURCE): 

If permission to reproduce is not granted to ERIC, or, if you wish ERIC to cite the availability of the document from another source, please 
provide the following information regarding the availability of the document. (ERIC will not announce a document unless it is publicly 
available, and a dependable source can be specified. Contributors should also be aware that ERIC selection criteria are significantly more 
stringent for documents that cannot be made available through EDRS.) 



Publisher/Distributor: 



Address: 



Price: 



IV. REFERRAL OF ERIC TO COPYRIGHT/REPRODUCTION RIGHTS HOLDER: 

If the right to grant this reproduction release is held by someone other than the addressee, please provide the appropriate name and 
address: 




V. WHERE TO SEND THIS FORM: 



Send this form to the following ERIC Clearin 




ERIC Clearinghouse for Co mmuni ty Colleges 
UCLA 

3051 Moore Hall, Box 951521 
Los Angeles, CA 90095-1521 
800/832-8256 
310/206-8095 fax 




However, if solicited by the ERIC Facility, or if making an unsolicited contribution to ERIC, return this form (and the document being 
contributed) to: 

ERIC Processing and Reference Facility 
IrtHnWest Street^^loor 
Laurel%(aryland^f0707-3598 



ERIC 



TelephprfS:^qi-497-4080 
Tgjtfree: 80oNfi-3742 
TAX: 301-953-04 
^-mail: ericfac@ineted!^ov 
WWW: http://ericfac.piccard.csc.com 



*^.-.1.1- 1 88 (Rev. 9/97) 

PREVIOl IS VERSIONS OE THIS EORM ARE ORSOI.ETE 



